Even when imaging the same object, there is a possibility that the object in the image is displaced due to a displacement in camera position and the like. In such a case, image alignment is performed by converting one image so that the object in the image is at the same location as the object in the other image. Methods for calculating a conversion parameter of conversion effecting such image alignment can be classified into a feature point-based method and a region-based method.
The feature point-based method is a method of extracting feature points from each of the two images subject to the alignment and associating the feature points between the images to calculate the conversion parameter.
The region-based method is a method of specifying such conversion that minimizes a difference in luminance (pixel value) between the images. A gradient method is known as the region-based method. The region-based method is also called the Lucas Kanade method.
The region-based method is described in Non Patent Literature (NPL) 1. In the method described in NPL 1, the conversion parameter is converged by repeatedly calculating a conversion parameter update and updating the conversion parameter. In NPL 1, conversion parameter update modes are classified into an additive algorithm and a compositional algorithm. The additive algorithm is an algorithm of updating the conversion parameter by addition of the conversion parameter update to the conversion parameter. The compositional algorithm is an algorithm of updating the conversion parameter by composition of the conversion parameter with the conversion parameter update.
In NPL 1, there is also classification into a forward algorithm and an inverse algorithm, in view of whether, in the case of converting one image to align it with the other image (template image), the conversion parameter update is derived based on the image to be converted or the template image. The forward algorithm is an algorithm of calculating the conversion parameter update based on the image to be converted. The inverse algorithm is an algorithm of calculating the conversion parameter update based on the template image.
Since each of the additive algorithm and the compositional algorithm can be classified into the forward algorithm and the inverse algorithm, four types of algorithms are available.
An image alignment method developed from the method described in NPL 1 is described in Patent Literature (PTL) 1. In the method described in PTL 1, when aligning the input image and the template image, a notice area is set in the template image, pixels smaller in number than all pixels in the notice area are selected from all pixels in the notice area, and the conversion parameter is estimated using only the selected pixels. A method of spatially thinning out pixels in the notice area is presented as a method for selecting pixels smaller in number than all pixels in the notice area from all pixels in the notice area. In more detail, a method of selecting spatially discrete pixels from the pixels in the notice area is presented. For example, a method of selecting pixels at intervals of 10 pixels from the pixels in the notice area is presented.
PTL 1 also describes selecting pixels based on partial derivatives of all pixels in the template image with respect to the deformation parameter. In more detail, PTL 1 describes that a partial derivative is determined for each pixel and pixels whose partial derivatives are higher than a threshold are selected, and also that a level of importance calculated from the partial derivative is determined for each pixel and pixels whose levels of importance are higher than a threshold are selected.
A pyramidal Lucas Kanade method is described in PTL 2. In the pyramidal Lucas Kanade method, alignment is possible even in the case where there is a large degree of deformation between the images to be aligned.